The progress in hardware and software has been activating the publication of electronic books as a new form of books in place of existing books on a paper medium. It has been realized to read cartoons and novels on a personal computer or a portable terminal.
Although these electronic books can be prepared correspondingly to what is called multimedia data, such as an audio, an image, a dynamic image, and an animation, it is costly and laborious to fabricate an electronic book as the primary object. Thus, an electronic book is fabricated frequently by directly computerizing an existing book.
There are following problems in the fabrication of an electronic book in accordance with a conventional art. A first problem exists in the case of converting, to a naturally monochromatic image, an image read by an image inputting means such as a scanner from a color-printed original or the like.
The conversion from a color image to a monochromatic image is generally performed by extracting the brightness component from the color image. The following Formula (1) is a formula for extracting the brightness component (Y) from red (R), green (G), and blue (B) components.Y=0.299R+0.587G+0.114B  (1)
This method is used also in the NTSC system which is a system for TV broadcasting, and is widely known. In the NTSC system, a color image is decomposed into a brightness signal and a color difference signal and then transmitted. Therefore, even a monochromatic television set can display a naturally monochromatic image on the television screen by receiving and reproducing the brightness signal alone.
Formula (1) depends on the characteristics of human visual sensation, and a full-color image such as a photograph can be converted to a naturally monochromatic image by converting the brightness component to the monochromatic image.
Meanwhile, the contents of an electronic book computerized directly from an existing book can be viewed on a color display without any problem because of the unlimited number of display colors. However, a portable terminal frequently uses a monochromatic liquid-crystal display because of the important factors of a low price and a low power consumption.
Accordingly, a full-color image such as a photograph is usually converted to a monochromatic image. Using Formula (1) to convert it to a naturally monochromatic image, monochromatic contents can be produced from an existing book. Or, a color image can be displayed on a monochromatic display.
However, in the case of a certain existing book, especially a cartoon journal, the image is sometimes printed in two colors with red ink and black ink or in a color with ink of a color selected from the group consisting of red, green, blue, and the like other than black. In that case, a desirable image quality is hardly obtained by converting such an image using Formula (1).
For example, in the case of an image printed in two colors with red ink and black ink, black is used for contours and shadows, and red is used for flesh color. When such an image is converted to a monochromatic image using Formula (1), the low mixing-ratio of red causes a problem that the red part becomes darker than the actual image.
Further, an image is sometime printed in a single color other than black, such as red, green, and blue. When people view such an image, the color itself is not explicitly recognized, and the resulted impression is similar to that of the image printed in a color of black. However, when such an image is converted to a monochromatic image using Formula (1) in a similar manner to that for a photograph, the resulted image has a thin color and a low contrast. In particular, an image printed with green ink results in an image with a very thin color because of the large mixing-ratio of green in Formula (1).
A second problem exists in the case of contracting a composite image such as a cartoon comprising a character, a line drawing, and a halftone dot, with good precision and without a moire.
Printed matter, such as a cartoon and a novel, is originally printed in a very high resolution, and a scanner for reading it also has a high resolution of 400 dpi (dot per inch), 600 dpi, or higher. In contrast, the display resolution of a personal computer or a portable terminal is at most about 100 dpi. Thus, the contraction of an image is necessary to display an existing book on a personal computer or a portable terminal.
In many cartoons, a halftone screen is used for the pseudo-expression of density and gradation. Since the halftone screen comprises fine mesh dots, lines, and patterns, the contraction thereof, as known, ordinarily results in a pattern of stripes or lattice which is called a moire. In the invention, a region to which the pseudo-expression of density or gradation is imparted is called a pseudo-density region.
Conventional art of contraction is classified into two major methods: a method in which the whole region is homogeneously processed and a method in which the region is divided and each of the divided regions is contracted optimally. The method in which the whole region is homogeneously processed generally includes a thinning-out process to achieve the contraction by simply thinning out the pixels and an averaging technique to determine the pixel value of the contracted image by averaging the pixel values of the original image (Hitoshi Kiya, Resolution Conversion of Digital Image, CQ Publishing, The Interface, June 1998, p. 72).
With regard to the region-dividing method, an image processing apparatus is described in Japanese Unexamined Patent Publication JP-A 4-365182 (1992). In accordance with the image processing apparatus described in JP-A 4-365182, a binary coded image is divided into two regions, i.e., drawing and painting regions. In the drawing region, contraction is performed so as to conserve the fine lines. In the painting region, a multi-value encoding process is performed on the base of the pixel density, and then a contraction/binary-encoding process is performed, thereby to contract even the character and the drawing sections with precision and without a moire.
However, in the case of contracting an image such as a cartoon comprising a character, a line drawing, and a halftone dot in combination, the thinning-out process causes a moire in the halftone section and unclearness and blurring in the character section and the line drawing section. In contrast, in the averaging technique, a moire is suppressed and the unclearness and blurring of a character and a fine line seldom occur, but the clearness of the whole is lost. To express the clearness, an edge enhancing process can be used after the contraction. However, the suppressed moire is also enhanced and appears. Although the moire can be completely eliminated by enlarging the area in which the pixels are averaged, characters and drawings blur on the other hand.
As mentioned above, prior art homogeneous processing by the thinning-out process or the averaging process cannot achieve clear characters and drawings without a moire. Thus, it is necessary to divide the image into regions and process each region appropriately.
The image processing apparatus described in JP-A 4-365182 performs region-dividing and performs a contraction process appropriate for each region. However, in a cartoon and the like, a line drawing may exist in halftone dots. Thus, a character and a line drawing cannot be separated by simple pattern matching. Further, a character exists in a balloon within the image. Thus, separation using a simple rectangle is difficult. In a method wherein an image is separated into two regions, i.e., a character/line drawing region and the other region, and wherein smoothing is performed in the region other than the character/line drawing region, an error frequently occurs in the extraction of the character/line drawing region. For example, a rather blurring part in a fine line and a complicated character is sometimes not extracted as a character/line drawing region because of the low edge component. The process of smoothing of this region causes further blurring in the character and the line drawing section.
A third problem exists in the case of performing the correction to eliminate a ghost image which occurs during the reading of an original printed on both sides or an original being stacked.
Since an existing book is ordinarily printed on both sides of a paper, there is a problem of what is called a ghost image, which is an image on the back side being seen through during the reading thereof with a scanner and the like or during the reading of a page being stacked with a scanner and the like.
An image reader described in Japanese Unexamined Patent Publication JP-A 6-14185 (1994) is a prior art apparatus for correcting a ghost image. The image reader described in JP-A 6-14185 eliminates a ghost image part, which has a low density, by reducing the density through the density correction of the image signal, thereby preventing the copying of the seen-through image of the back side of an original or the next page of a stacked original.
An image forming apparatus described in Japanese Unexamined Patent Publication JP-A 6-62216 (1994) is an apparatus for correcting a ghost image using front image data and back image data. The image forming apparatus described in JP-A 6-62216 performs an AND operation between the front image data and the back image data, performs the smoothing of the output by a histogram calculation, and then performs a threshold process. Then the apparatus combines it with image data which is the front image data subtracted by the data of the superposed part, thereby eliminating the ghost image without the loss of the low density part of the front image.
Further, an image processing apparatus with ghost-image eliminating function described in Japanese Unexamined Patent Publication JP-A 8-340447 (1996) eliminates a ghost image by detecting a ghost image region in a video signal and the ghost image level within the ghost image region and by correcting the density of the ghost image region using the ghost image level.
However, since the image reader described in Japanese Unexamined Patent Publication JP-A 6-14185 (1994) performs the density correction on the whole image, there occurs a problem that a halftone section whitens out and that a character becomes unclear.
With regard to the image forming apparatus described in JP-A 6-62216, a ghost image cannot be completely eliminated in some cases, for example, in the case where a halftone section is seen through as a ghost image. Further, the positional relationship between the front image and the back image needs to be previously known, but the positions in which the images are read are not necessarily identical even if automatic paper feeding is used. Thus, a ghost image cannot be completely eliminated in such a case where the images are shifted from a predetermined position.
In the image processing apparatus with ghost-image eliminating function described in JP-A 8-340447, a ghost image region is defined as a region in which non-character pixels are successive, a region in which non-painting pixels are successive, a region in which pixels having a density lower than or equal to a predetermined density are successive, and a region in which pixels having a saturation lower than or equal to a predetermined saturation are successive. Accordingly, the determination is carried out in a small area. Therefore, a ghost image cannot be separated from a halftone image on the front side, for example, when the ghost image is caused by a widely spreading black dark region of characters or images.
A fourth problem exists in the case of performing the aligning between desired pages in the image data of a book inputted in an authoring system for preparing the contents of an electronic book by inputting the book on the image base.
Although an electronic book can be prepared correspondingly to what is called multimedia data, such as an audio, an image, a dynamic image, and an animation, it adopts a text (character-code) based format. Meanwhile, books on a paper medium, what is called “books” are presently being published at a pace of 500,000 or more titles a year. The accumulated number is huge. However, the number of the computerized titles is very few and almost all exist only on a paper medium Such a prior art electronic book has the following problems because of adopting the text-(character-code) based format.
For the authoring of a book on a paper medium, the text data needs to be prepared by human work or with an OCR. Thus, the preparation of the contents requires a lot of time, which causes a difficulty in the timely supply of contents in a large amount. Further, it is difficult to prepare the contents of a book such as a cartoon and a photographic journal, in which a majority of the data is non-text. For this reason, present number of the electronic book contents is as small as a few hundred, and many of the contents are dictionaries. Accordingly, the share of electronic books is presently below 1% of that of paper books. In particular, the small number of the contents is a fatal problem, which is significantly preventing the spread of electronic books. In such a situation, to resolve the above-mentioned problems, it is possible to obtain electronic book contents by inputting them on the image base. This has the following advantage.
Contents preparation can be carried out basically only by scanning an existing book on a paper medium, which permits to supply a large amount of the contents in a short term. It permits to supply the contents of a cartoon, a photographic journal, and the like, which was impossible in a text-based electronic book. Inputting is easy even when a book, such as an old document, contains a character not in the present character code system, for example, an external character and a heteromorphic character. Overseas deployment (spreading) of a viewer and a total authoring system is easy because of the independence of a language (character code). By the advantage mentioned above, an electronic book on the image base resolves all problems in an electronic book on the text base. To obtain the electronic book contents by inputting it on the image base, it can be inputted by scanner inputting with an ADF (auto document feeder) and the like, and various processes such as document structuring are carried out on it. However, the following problems occur.
In the case where an image is inclined or shifted during the scanner inputting, an user of the electronic book feels uncomfortableness because the inclination is emphasized more than in the case of a paper book by a reference line (for example, an edge section of a CRT and a liquid-crystal screen) existing in a viewer. Thus, the process of correcting this is necessary. Manual processing of this needs a lot of working time, which causes a substantial increase in the authoring time. In particular, inclination and shift in a main page results in a strong uncomfortableness during the viewing on a viewer. Further, the checking of all pages for the revision of the electronic book contents causes an increase in the authoring time, which prevents to supply a large amount of contents in a short term. Thus, an appropriate process is necessary.
A first object of the invention is to provide an image processing apparatus, an image processing method, and a medium on which an image processing program is recorded, each for converting an original printed in two colors or a color other than black to a naturally monochromatic image.
A second object of the invention is to provide an image processing apparatus, an image processing method, and a medium on which an image processing program is recorded, each for contracting a composite image such as a cartoon comprising a character, a line drawing, and a halftone dot, with clearness and without a moire.
A third object of the invention is to provide an image processing apparatus, an image processing method, and a medium on which an image processing program is recorded, each for correcting an image to eliminate a ghost image which occurs during the reading of an original printed on both sides or an original being stacked.
A fourth object of the invention is to provide an image processing apparatus, an image processing method, and a medium on which an image processing program is recorded, each for obtaining aligned images.